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ABSTRACT 

A study of a continuous quality impressment effort at 
Auburn University (Alabama) is reported. The study investigated 
variables predicting which students are at risk of failure. Data were 
drawn from the records of pre-engineering students enrolling at the 
university in fall terms of 1991 and 1993-95 (n= : 2,505). Data sources 
included achievement tests, high school transcripts, several 
standardized psychometric tests, freshman survey results, college 
grades, and exit questionnaire responses. Results are presented 
concerning the proportion of white and black pre-engineering students 
making a successful transition to the engineering program, 
correlation between college entrance examination scores and college 
grades/prograra completion, the predicted impact of raising entry 
standards, racial differences in achievement in pre-engineering, sex 
differences in achievement, correlation of personality factors and 
achievement, relationship of persistence to student use of time, 
program elements that students liked or disliked, and reasons offered 
for leaving engineering. Results of a survey of students 
(undergraduate and graduate) and faculty in the college of 
engineering, designed to identify factors important in improving the 
program (curriculum, accreditation, financial support, 
post-graduation employment, group working experiences, course 
attributes, cooperative experience, instruction) are then summarized. 
(MSE) 
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The focus of this research is on assessment aimed at student 
retention as well as on quality improvement throughout the higher 

educational experience. 



ASSESSMENT AIMED AT STUDENT RETENTION 



Objectives 

/ 

Specific program objectives are to predict which students are at 
risk of failure and to develop intervention strategies that will 
reduce failure of at-risk students. 

Data Base 

The data base is from an ongoing longitudinal study of student 
retention in engineering beginning with data from 1991. Included 
to date are students entering Auburn University as pre-engineers 
fall, 1991 (N = 837); fall, 1993 (N = 547); fall, 1994 (N = 531); 
and fall, 1995 (N = 590) . 

Sources of Data 



Sources of data were achievement tests, high school transcripts, 
the Myers-Briggs Personality Type Indicator, the College Student 
Inventory, Group Embedded Figures Test, College Freshman Survey, 
college grade reports, and an exit questionnaire. 

Preliminary Results of Assessment Aimed at Student Retention 

1. Of the 837 students who began as freshmen in the pre- 
engineering program in 1991, only 53.6% successfully made the 
transition tc engineering. Furthermore, there were 91 black 
students in the pre-engineering group, and only 33% (30) 
students made the transition to engineering. Hence, one must 
conclude that the attrition of students in engineering is 
quite high, but the attrition of black students is much higher 
than that of their white counterparts. 

2. In general, there is a direct correlation between ACT scores 
and grades students received in a particular course. For 
example, the mean ACT composite score for the 29 students from 
the 1991 freshman class who received an F in Computer Science 
was 21.8, while the mean score for the 84 students who 
received an A was 27.6. A similar relationship holds for the 
mean ACT math score. 
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There appears to be a strong correlation between ACT scores 
and successful completion of the pre-engineering program. The 
correlation coefficient for the ACT composite score was .34. 
For the ACT math score, it was .38. The mean ACT composite 
score for students who successfully completed the pre- 
engineering program was 26.27, and the mean ACT math score was 
26.45. 

4. Comparing the total number of 1991 freshman students wit'i an 
ACT composite score greater than or equal to a particular 
value with the number of students who successfully completed 
the pre-engineering program shows that almost 71% of the 
students who entered with a score of 27 successfully completed 
the pre-engineering program, while only 60% of the students 
who entered with a score of 24 were successful. 

5. The impact of raising the minimum ACT composite score for 
admission to pre-engineering would have a devastating effect 
on minority students. Data for the 1991 pre-engineering 
students indicate that the mean for white students was 25.6 
and the mean for black students was 21.8. The score 
distribution was such that 86% of the white students had ACT 
composite scores higher than the 21.8 mean value for black 
students while only 14% of the white students had scores lower 
than the 21.8 mean. These data suggest that academic 
potential may be one factor which must be considered when 
seeking solutions to the problem of high attrition for black 
students in engineering. 

6. To determine if factors other than academic potential 
contribute to the retention problem of black students in pre- 
engineering, black students and white students were matched on 
the basis of ACT composite scores using a random number 
generator to select the white students. Eighty-one students 
in each respective group were compared. Almost 46% of the 
white students successfully completed pre-engineering while 
only 33% of the black students successfully completed pre- 
engineering. Hence, it must be concluded that factors other 
than academic potential, as measured by the students' 
performance on the ACT, play a role in determining if black 
students will be able to complete pre-engineering 
successfully. 
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A very strong relationship (correlation coefficient of *55) 
existed between first quarter GPA and successful completion of 
the pre-engineering program. For the 1991 pre-engineering 
students, the mean GPA for those admitted to engineering was 
3.0 while that for students not admitted because of grades 
was' 1.89. The mean first quarter GPA for those students not 
admitted to engineering but with overall GPAs equal to/greater 
than 2.2 was 2.0. Hence, it appears that there is little 
difference in the overall grades of students not admitted to 
engineering with grades less than 2. '2 and those not admitted 
with grades equal to/greater than 2.2. 

The eight pre-engineering courses with the lowest mean grade 
which had enrollments of 100 students or more were Chemistry 
102, Chemistry 101, Physics 220, History 101, Chemistry 104, 
History 102, Chemistry 103, and Math 160. The mean grade for 
these courses ranged from a low of 1.5 for Chemistry 102 to a 
high of 2.2 for Math 160. In all of these courses, white 
students performed better than black students. Because of the 
diverse nature of these courses, these findings suggest that 
bridge or academic assistance programs limited to math and 
science courses might be too narrow an approach for effective 
intervention. 

9. For the 1993 pre-engineering student group as a whole, strong 
correlations were found between ACT composite scores and first 
and second quarter GPAs and also between ACT math scores and 
first and second quarter GPAs. The correlations were also 
quite strong when the students were classified on the basis of 
sex. When the 1993 students were classified on the basis of 
race, very strong correlations were observed for white 
students, but the correlations did not hold for the black 
students. For example, the correlation coefficient for the 
relationship between first quarter GPAs for white students and 
their ACT math scores was .53. However, for black students 
the coefficient value dropped to .06. 

Care should be taken in comparing the correlations for the 
different racial groups because only 33 black students were 
included in the 1993 data base. When data from the 1991 
freshman class which contained 91 black students were 
analyzed, the correlation coefficient for the relationship 
between first quarter GPA and ACT math score was .42. 
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10. Except for the Thinking/Feeling preference scale on the Myers- 
Briggs Type Indicator, the 1993 pre-engineering students were 
fairly equally divided relative to their preference on each 
MBTI preference scale. For the Thinking/Feeling preference 
scale, approximately 64% of the students were classified as 
thinking while 36% were classified as feeling. 

11. Attempts to correlate the first and second quarter GPAs for 
the 1993 pre-engineering students as a group with the MBTI 
preference scales were, in general, unsuccessful. The 
strongest correlation observed was —.14 between the second 
quarter GPA and the Perceiving/ Judging scale. However, when 
the correlations were computed for males and females and for 
different racial groups, a number of fairly strong 
relationships were seen. 

For female students the correlation was -.21 between second 
quarter GPA and the Extravers ion/ Introvers ion scale and -.24 
between second quarter GPA and the Perceiving/ Judging scale. 
No correlation this strong was found for male students. The 
strongest relationship noted for this group was -.11 between 
second quarter GPA and the Thinking/Feeling scale. 

For black students the strongest relationships were noted 
between first quarter GPA and the Extraversion/ Introversion 
scale (r = -.20) and between first quarter GPA and the 
Thinking/Feeling scale (r = .23). The strongest relationship 
noted for white students was between second quarter GPA and 
the Perceiving/ Judging scale (£ = ^.12). 

12. Correlations for the Group Embedded Figures Test administered 
to the 1993 pre-engineering students and first and second 
quarter GPAs were .11 and .10 respectively. 

13. A number of strong correlations were observed between 

responses to questions on the College Student Inventory and 
first and second quarter GPAs. The strongest relationships 
were observed for the predicted academic difficultly scale (jr 
= -.34 and -.31 respectively), the student's assessment of 
his/her high school performance (£ = -.44 and -.48 

respectively) , and the student's assessment of his/her 
composite score on the ACT (r = .41 and .34 respectively). 
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Multiple regression analysis showed a strong relationship 
(regression coefficient of .61) between first quarter GPA and 
ACT math scores, self-reported high school grades, the study 
habits scale of the CSI, scores on the Group Embedded Figures 
Test, the highest educational level of the student's father, 
and the student's self-rating of his/her academic preparation. 

A similar analysis for the second quarter GPA (regression 
coefficient of .57) showed the important independent variables 
to be ACT math scores, self-reported high school grades, 
scores on the study habits scale of the CSI, and the highest 
educational level of the student's mother. 

15. Black/white comparisons on the overall grade index from the 
high school transcripts of 1991 freshmen pre-engineering 
students revealed no significant differences. According to 
these high school transcript data for the freshmen pre- 
engineering students in the fall of 1991, the blacks made 
grades as good as those of the whites in high school. 

16. There were no significant differences in the amount of time 
persisters and dropouts from engineering spent in class, 
studying, watching television, or working at a place of 
employment. Persisters did tend to spend more time in 
nonacademic university activities (e.g., fraternity/sorority , 
intramural sports, clubs, band, etc.). 

17. Regarding ratings of instruction in mathematics, science, and 
other courses, students tended to see their professors as 
being organized, genuinely interested in teaching, and willing 
to help students individually. Mean composite ratings across 
nine items by persisters and dropouts were not significantly 
different for mathematics and science courses, but the 
composite mean rating by dropouts of other courses was 
significantly higher than that by persisters. 

18. When dropouts were asked what they had liked while in 
engineering, 16.13% of the blacks as compared to 4.48% of the 
whites indicated that they liked the help or tutoring 
available. A higher percentage of whites (17.24%) indicated 
that they liked math than was true for the blacks ( 12 . 90%) . 
Also worthy of attention is the discrepancy in the percentages 
of whites (15.52%) and blacks (9.68%) who cited concern and 
attention demonstrated by faculty and counselors as something 
they liked. 
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19. When dropouts were asked what they had not liked while in 
engineering , the percentage of blacks (25.81%) who indicated 
that they did not like mathematics was substantially higher 
than the percentage of whites (14.48%). Also a much higher 
percentage of blacks (22.58%) than of whites (13.79%) 
indicated that they disliked physics. 

20. Poor grades were given as the reason for leaving engineering 
by an overwhelming percentage of the black students (41.94% as 
compared to 23.79% of the white students). Other reasons for 
leaving engineering cited by a number of the black students 
were that they preferred another major (25.81%) , did not like 
engineering or lost interest (16.13%), or had problems with 
specific courses (12.90%). 



8 




d 



QUALITY IMPROVEMENT 



College Goal 

The initial goal was to develop a means for defining and assessing 
quality as it relates to instructional, research, and outreach 
programs. A subsequent goal is to employ results to guide 
continuous quality improvement in the College of Engineering. 

General Process 

In October of^993, the College of Engineering began to investigate 
how to integrate the philosophy of continuous quality improvement 
into its instructional program. A Quality Standards Committee was 
formed. One of their early undertakings was to determine how well 
those constituents who derive a benefit or advantage from an 
activity or process carried out under the auspices of the College 
of Engineering feel that their needs are being met. 

Constituents 



Constituent groups were students (both undergraduate and graduate) , 
faculty, industry, and alumni. Initial data collections were 
limited to these groups. 

Sources of Data 



Each constituent group provided data via questionnaires or 
telephone interviews. Students and faculty were asked to judge 
which attributes were important to the quality of engineering 
programs and at what level their expectations were being met 
relative to implementing each attribute. Alumni were asked to 
provide information on the importance of instructional program 
attributes and how well their expectations were met for those 
attributes. Business and industry representatives were asked to be 
sources of information on important attributes for program 
graduates as well as for information on employment. Results from 
the survey of undergraduate students, graduate students, and 
faculty in the College of Engineering are being reported here. 

Student/ Faculty Results Related to Quality Improvement 

1. The most important attribute identified by each group was 
different. The undergraduate students consider "a curriculum 
that prepares one to pass professional licensure examinations' 1 
to be the most important attribute of the instructional 
program. Graduate students consider "support for graduate 
teaching and research assistantships" *~o be the most important 
attribute, while the faculty ionsider "engineering 
accreditation" to be the most importa it attribute. 
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2. A significant difference of opinion exists between the 
undergraduate students and the faculty relative to the 
importance of a curriculum that prepares one to pass 
professional licensure examinations. The undergraduate 
students consider this attribute to be the most important of 
the instructional program while the faculty place it in the 
bottom third of their attribute list relative to importance. 

3. All groups consider engineering accreditation to be a very 
important attribute. The faculty rate this attribute as the 
most important. The undergraduate students rate it as the 
second most important, while the graduate students rate it as 
the fifth most important. All three ratings are in the top 
third of each respective attribute list relative to 
importance. 

4. Financial support is, in general, an important issue for all 
groups of respondents. Every attribute listed in the 
Financial Support category for undergraduate students and 
graduate students is rated in the top one third of all 
attributes for importance. Four of the six attributes in the 
Financial Support category of the faculty survey are also 
rated in the top one third of all attributes for importance. 

5. Employment after graduation is an important concern for all 
groups of respondents. "Graduates who are sought by potential 
employers" is rated the third most important attribute by the 
faculty and graduate students and the eleventh most important 
attribute by undergraduate students. 

6. Group working experiences do not appear to be highly valued by 
any of the respondent groups. The undergraduate students 
place only two of the five attributes listed in this category 
within the top one third of all attributes for importance. 
Graduate students place only one of the foul attributes listed 
in the Group Working Experience category in the top one third 
of their attributes list, while the faculty place none of the 
three attributes in this category in the top one third of 
their attributes list. 
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7 . The two attribute categories which are valued the least by the 
respondent groups are Flexible /Relevant Curriculum , and 
Satisfying Campus Experience . The undergraduate students and 
the graduate students rate no attribute in either category at 
an importance level that would place it in the top one third 
of the attribute list for importance. The faculty rate one 
attribute in each category high enough to be in the top one 
third of the attribute list for importance. These attributes 
are "courses which are designed to develop students' oral and 
written communication skills" (rated number 10) for the 
Flexible /Relevant Curriculum category and. "safe campus 
environment" (rated number 2) for the Satisfying Campus 
Experience category . 

8. Undergraduate students rate the attribute, "courses which are 
designed to develop students' oral and written communication 
skills," next to last in their attribute list for importance. 

9. Neither the faculty nor the undergraduate students seem to 
consider that a co-op experience is of high value. 
Undergraduate students rate the attribute, "opportunity for 
co-op experience," 28th in their attribute list for 
importance, while the faculty rate it 23rd. 

10. The graduate students do not seem to be particularly motivated 
co pursue a career in academia. They rate the attribute, "a 
curriculum that prepares one for a position in academia," 41st 
in their attribute list for importance. 

11. Faculty do not appear to feel that it is important to place 
additional emphasis on the instructional program. The 
following attributes were rated so they fell in either the 
middle or lower one third of the attribute list relative to 
importance : 

— "Outstanding teaching given the same weight as 
outstanding research in tenure, promotion, and salary 
adjustment deliberations." 

— "Faculty teaching awards available to reward 
outstanding instruction." 

— "Research and publishing in the teaching area given the 
same consideration as similar activities in other areas." 

— "Internationally recognized teaching 1 culty." 
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